Variations on the Discount Prior Method: Dynamic Borrowing from a Historical Study

July 23, 2018

Thanks to my co-authors:

Tarek Haddad, Medtronic
Xuefeng Li, CDRH/FDA
Ram Tiwari, CDRH/FDA
Rajesh Nair, CDRH/FDA
Jianxiong Chu, CDRH/FDA

Outline

Review of Conditional Power Prior

Prior Distribution given \(\scriptsize{D_0}\)

\[\color{black}{\large{ {\pi ^{CPP}}(\theta |{D_0},{\alpha _0}) \propto {L_0}{(\theta |{D_0})^{{\alpha _0}}}\pi (\theta )}} \]

Posterior Distribution given \(\scriptsize{D}\)

\[\color{black}{\large{\pi (\theta |D,{{D}_{0}},{{\alpha }_{0}})\propto {{L}_{0}}{{(\theta |{{D}_{0}})}^{{{\alpha }_{0}}}}\pi (\theta )L(\theta |D)}}\]

Prior with two prior datasets \(\scriptsize{D_0}\) and \(\scriptsize{D_1}\)

\[\color{black}{\large{\begin{align} & {{\pi }^{CPP}}(\theta |{{D}_{0}},{{D}_{1}},{{\alpha }_{0}},{{\alpha }_{1}})\propto \\ & \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{{L}_{0}}{{(\theta |{{D}_{0}})}^{{{\alpha }_{0}}}}\pi (\theta ){{L}_{1}}{{(\theta |{{D}_{1}})}^{{{\alpha }_{1}}}} \\ \end{align}}}\]

Discount Prior Method (Haddad et al., 2017)

If we want the amt of borrowing to be based on similarity between prior and curr data sets, then we can't use a pre-specified power parameter value because the current data are not known.

So we use dynamic borrowing, where a0 is based on the relationship between the prior and current data responses after the current data are observed. So now the power parameter is a function of D and D0.

But, what is pre-specified is the form of the relationship.

In the 2017 paper, we defined this form by a weibull CDF of \(p\).

where if \(\theta\) defines a failure rate, say, p is defined as prob(t < t0)…

So, if p is large, you would borrow more from the prior data bc a0 is closer to 1. And, borrowing is conservative because p is the prob that the prior failure rate is higher than the current rate.

You can have a two-sided discount fcn that prevents you from borrowing when prior rate is much worse.

Prior given \(\scriptsize{D_0}\) (and \(\scriptsize{D}\))

\[\color{black}{\large{\pi ^{CPP}(\theta |{D_0},D) \propto L(\theta |{D_0})^{\color{red}{\alpha_{0}(D_0,\,D)}\pi (\theta )}}}\]

\[\large{\alpha_{0}(D_0,\,D) = 1 -\exp(-(\frac{p}{\lambda})^k) = F(p|\lambda, k)}\]

where

\(\large{p = P({\theta }<{{\theta }_{0}}|\,{{D}_{0}},\,{D})}\)

\(\,\,\, \large{\theta}\) ~ \(\large{\pi(\theta | D) \propto L(\theta | D)\pi(\theta)}\)

\(\large{\theta_0}\) ~ \(\large{\pi(\theta | D_0) \propto L(\theta | D_0)\pi(\theta)}\)

Comments on Discount Prior Method

The current data are used twice, both in the prior and in the likelihood to obtain the posterior. This violates the Likelihood Principle.
Effective prior sample size can change after seeing the current data. It is not legitimate Bayesian because the prior is fully determined a posteriori.
A "stochastic ordering" measure of similarity may be influenced by the current data sample size so that more prior information is borrowed when the sample size is smaller.

Potential Variations to Discount Prior Methodology

Determining the power parameter from an external existing dataset \(\scriptsize{D_1}\)

Eliminates double use, with \(\scriptsize{\alpha_0}\) determined a priori

Conditional Power Prior becomes:

\[\color{black}{\large{{{\pi }^{CPP}}(\theta |{{D}_{0}},{{D}_{1}})\propto {{L}_{0}}{{(\theta |{{D}_{0}})}^{{{\alpha }_{0}}( {{D_0}},\, \color{red}{{D}_{1}} )}}\pi (\theta )} }\]

\[\color{black}{\,\,\,\,\,\,\,\,\,\,\large{0\le {{\alpha }_{0}}({{D}_{0}},{{D}_{1}})\le 1}}\]

\(\scriptsize{{{\alpha }_{0}}({{D}_{0}},{{D}_{1}})}\) is the proportion of down-weighting of the LH of \(\scriptsize{D_0}\) using information from both \(\scriptsize{D_0}\) and \(\scriptsize{D_1}\).

Determining the power parameter from interim current data (\(\scriptsize{D_1}\))

\(\scriptsize{{{\alpha }_{0}}({{D}_{0}},{{D}_{1}})}\) may not reflect a comparison between prior data (\(\scriptsize{D_0}\)) and current data, and \(\scriptsize{D_1}\) may not exist.

An alternative is to use interim information from the current data.

Suppose \(\scriptsize{D_1}\) is an initial subset of \(\scriptsize{D}\) of size \(\scriptsize{n_1}\) = %\(\scriptsize{n}\).
Let \(\scriptsize{D=\left[{{D}_{1}},{{D}_{2}}\right]}\), then the likelihood of \(\scriptsize{D}\) is

\[\small{L(\theta |D)=L(\theta |{{D}_{1}})\times L(\theta |{{D}_{2}})}\]

Mathematically, we can consider \(\scriptsize{D_1}\) as a "second" prior dataset because it comes before \(\scriptsize{D_2}\) and after \(\scriptsize{D_0}\).

Determining the power parameter from interim current data (\(\scriptsize{D_1}\))

The resulting prior conditions on \(\scriptsize{D_0}\) and \(\scriptsize{D_1}\), but only down-weights the LH for \(\scriptsize{D_0}\).

\[\color{black}{\large{\begin{align} & {{\pi }^{CPP}}(\theta |{{D}_{0}},{{D}_{1}})\propto {{L}_{0}}{{(\theta |{{D}_{0}})}^{{{\alpha }_{0}}({{D}_{0}},\color{red}{{D}_{1}})}}\pi (\theta ) \color{red}{L(\theta |{{D}_{1}})} \\ \end{align}}}\]

Posterior given current data \(\scriptsize{D_2}\):

\[\color{black}{\large{\begin{align} & \pi (\theta |{{D}_{0}},{{D}_{1}},{{D}_{2}})\propto \\ & \,\,\,\,\,\,\,\,\,\,\,\,\, L{{(\theta |{{D}_{0}})}^{{{\alpha }_{0}}({{D}_{0}},{{D}_{1}})}}\pi (\theta )L(\theta |{{D}_{1}}) \color{red}{L(\theta |{{D}_{2}})} \\ \end{align}}}\]

Yesterday's posterior is today's prior

Determining the power parameter from interim current data (\(\scriptsize{D_1}\))

Things to note:

\(\scriptsize{{{\alpha }_{0}}({{D}_{0}},{{D}_{1}})}\) is not a function of the entire current data \(\scriptsize{D}\). The most recent data \(\scriptsize{{{D}_{2}}}\) is not included in the prior.
Modified CPP is a legitimate prior on \(\scriptsize{\theta}\).
We do use the subset \(\scriptsize{{{D}_{1}}}\) twice for making inference on \(\scriptsize{\theta}\)
If \(\scriptsize{{{D}_{1}}}\) were small enough to be considered not essential for inference, we could eliminate \(\scriptsize{L(\theta |{{D}_{1}})}\) from the posterior. However, we never want to discard any current data.

Measuring Similarity of Prior and Current Data (Stochastic Ordering measure)

Another variation to the discount prior came after we discovered a scenario where the measure of similarity using a stochastic ordering depended on posterior variance, and could be anti-conservative ..To illustrate, I use a success rate definition for \(\scriptsize{\theta}\).

So supp the \(\scriptsize{\theta}\) that generated the curr data is better than the value, \(\scriptsize{\theta_0}\) that generated the prior data (on x-axis, we have \(\theta\) to the right of \(\theta_0\)). With vague priors, the post dist of \(\scriptsize{\theta}\) given the curr data and of \(\scriptsize{\theta_0}\) given the prior data might look like fig, which shows hyp post dists using the prior or curr data (with diff colors to indicate diff potential post vars given diff curr samp sizes).

Now, here, if we compute the post pr(\(\scriptsize{\theta}\) > \(\scriptsize{\theta_0}\)) shaded in fig, it will likely be high, regardless of the post var of \(\scriptsize{\theta}\), and, it's higher when post var is lower (curr samp size is larger).

Page down….

Hypothetical Posterior Distribution of \(\scriptsize{\theta}\) and \(\scriptsize{{{\theta }_{0}}}\) for Success Rate

Current data better than prior data

Shaded is \(\color{black}{\scriptsize{P(\theta >{{\theta }_{0}}|{{D}_{0}},D)}}\)

Measuring Similarity of Prior and Current Data (Stochastic Ordering measure)

Hypothetical Posterior Distribution of \(\scriptsize{\theta}\) and \(\scriptsize{{{\theta }_{0}}}\) for Success Rate

Current data better than prior data

Shaded is \(\color{black}{\scriptsize{P(\theta >{{\theta }_{0}}|{{D}_{0}},D)}}\)

Borrow more with
higher current data
sample size

Hypothetical Posterior Distribution of \(\scriptsize{\theta}\) and \(\scriptsize{{{\theta }_{0}}}\) for Success Rate

Current data worse than prior data

Shaded is \(\scriptsize{P(\theta >{{\theta }_{0}}|{{D}_{0}},D)}\)

So, the result is that we borrow more from the prior when the current data sample size is smaller. This appears anti-conservative because with a smaller sample size, the prior (which is better) could overwhelm the current data, leading to inference that more reflects the better prior.

From my simulations, it appears to be an issue mostly when the current data are just a little worse than the prior data. If there's time I can show you. END END

It's not clear that this is anti-conservative because we still have the full amt of data for inference.

And, even if we discard D1 after using it, we would be discarding less data if we borrow more prior data. So, ultimately inference might not be affected. I examine a limited number of cases in the simulations, but haven't come to a conclusion.

Using a two-sided disc dcn doesnt help. With a two-sided fcn, the power parm increases with “similarity” up to a point, then decreases in a mirror image. For the scenarios here, bc the shaded areas are less than 50-percent and therefore not on the mirror image decline, the smaller samp size will still borrow more than the larger size.

Hypothetical Posterior Distribution of \(\scriptsize{\theta}\) and \(\scriptsize{{{\theta }_{0}}}\) for Success Rate

Current data worse than prior data

Shaded is \(\scriptsize{P(\theta >{{\theta }_{0}}|{{D}_{0}},D)}\)

Borrow more with
lower current data
sample size
(anti-conservative?)

Alternative Measure of (Dis)similarity:
Kolmogorov-Smirnoff Statistic (KS)

\(\color{black}{\small{D_{n,m} = \sup\limits_{x}|D_{0,n}(x) - D_{1,m}(x)| = KS}}\)

Simulation Study

We ran a simulation study to evaluate these variations to the discount prior: such as using a percentage of D (referred to as D1) to help determine a0, where the percentage can be less than 100%, and using the KS similarity measure.

We simulated a normally distributed prior data set (D0) with mean 0 and variance 1, and sample size n0= 50 or 100.

For current data generation, we selected Normal mean values, \(\theta^{*}\), ranging from – 1 to 0, in increments of 0.25. For each value of \(\theta^{*}\) we simulated 15,000 current data sets (D) from a normal distribution with mean \(\theta^{*}\), variance 1, and size n = 50 or 100.
The percentage of D used to form D1 varied from 100% to 25%, in increments of 25%.
similarity measures:
We examine both type I error rate and power using these settings.

Prior data: \(\scriptsize{D_0 \text{~} N(0,1)}\) with sample size \(\scriptsize{n_0}\) = 100 (or 50).

Current data: Normal mean values, \(\scriptsize{{{\theta }^{*}}}\): \(\scriptsize{(– 1, – 0.75, – 0.5, }\) \(\scriptsize{– 0.25, – 0.1, 0)}\).

For each \(\scriptsize{{{\theta }^{}}},\) we simulated 15,000 current data sets D from a \(\scriptsize{N({{\theta }^{}}, 1)}\) distribution with size \(\scriptsize{n}\) = 100 (or 50).

The percent of \(\scriptsize{D}\) used to form \(\scriptsize{D_1}\): \(\scriptsize{100\%, 75\%, 50\%, 25\%}\).

Similiarity measure: KS statistic or stochastic ordering measure.

Simulation Study

Operating Characteristics: Type I error rate is computed under \(\scriptsize{H_0: \theta \le l}\), where \(\scriptsize{l \in (-1,..., -0.25, -0.10)}\).

\(\scriptsize{H_0}\) is rejected if posterior probability of \(\scriptsize{H_0}\) is less than 0.025.

Power is computed under \(\scriptsize{H_0:\theta \le -0.5}\) with values of \(\scriptsize{{{\theta }^{*}} \in (-0.35, -0.25, -0.10, 0)}\).

The maximum value of \(\scriptsize{\alpha_0}\) allowed is 1.0.

Form of \(\scriptsize{{{\alpha }_{0}}({{D}_{0}},{{D}_{1}})}\) for Simulations

\(\scriptsize{ 1 - exp(-(\frac{1-KS}{12})^{0.9})}\)

\(\scriptsize{1 - exp(-(\frac{p}{3})^{0.65})}\)

Weibull parameters should be pre-specified at the design stage.

\(\scriptsize{\color{black}{\alpha _0(D_0,D_1)}}\) by \(\scriptsize{{{\theta }^{*}}}\) (prior mean = 0 and \(\scriptsize{\color{black}{n_0 = n} = 100}\))

Figure shows \(\scriptsize{\alpha_0}\) (borrowing fraction) averaged over the 15,000 data sets, for each \(\scriptsize{\theta^{*}}\) value, when n0 = n= 100. Different colored lines are for the different percents of D that represent D1. Each panel shows the different similarity function used, KS or Stochastic ordering.

For KS measure, as \(\scriptsize{\theta^{*}}\) gets closer to the prior mean, we borrow more (\(\scriptsize{\alpha_0}\) increases). Also, we borrow more when we use more current data (% is higher), to determine a0.
For the stochastic ordering discount function, when \(\scriptsize{\theta^{*}}\) is less than prior mean of 0 (curr data worse than prior), we borrow more when we use less current data to determine a0 (so, 25% has the highest borrowing average, and 100% has the lowest). But this reverses when the current data mean is better than the prior mean. Also, because we have a one-sided discount function, the amt of borrowing doesn't fall again as it does for the KS similarity.

Type I Error Rate by \(\color{black}{\scriptsize{{{\theta }^{*}}}}\) (null value) when \(\color{black}{\scriptsize{n = n_0 = 100}}\)

Here, I present type I error rate by \(\color{black}{\scriptsize{\theta^{*}}}\) when it is consistent with H0. And l ranges from – 1 to –0.10.

LHS shows type I error rate when we dont discard D1 after using it in \(\scriptsize{\alpha_0}\), and the RHS shows when we do discard D1. Top panel shows the KS similarity measure, bottom panel shows stochastic ordering.

For the KS similarity measure we borrowed more from D0 (which is consistent with H1) with more of D used in the borrowing fraction. So, type I error rate tends to increase with a higher % of D being used in \(\scriptsize{\alpha_0}\), despite D coming from \(\scriptsize{H_0}\). So, the 100% line has the highest type i error rate.

For the stochastic ordering, because we borrowed more from the prior data with a smaller percentage of D used in a0 when the current data mean was worse than D0, we have a higher type I error rate for the lower % lines. But, despite borrowing more, type I error is very low, especially compared to the KS similarity measure with its discount function.

Now - RHS: where we discard D1 after using it to determine the borrowing fraction. Going from left to right, the a0 value remains the same; so you aren't borrowing a different amt of prior data. Therefore, you would expect that if you discard data from the null, you would make more type I errors than if you kept it. So, the error rate should be higher on the right (discarding).

This is mostly the case here: average type I error rate is higher on the right than the left. If it isn't the case, we found that it may be due to higher posterior variance from discarding data pushing a good portion of the posterior distn into the null territory. So, you don't reject the null as much, and so error rate is lower.

Type I Error Rate by \(\color{black}{\scriptsize{{{\theta }^{*}}}}\) (null value) when \(\scriptsize{\color{black}{n = n_0 = 100}}\) and max \(\scriptsize{\alpha_0(D_0, D_1) = 0.5}\) using KS

Power by \(\scriptsize{{{\theta }^{*}}}\) when \(\scriptsize{\color{black}{n = n_0 = 100}}\)

Posterior SD of \(\scriptsize{\color{black}{\theta }}\) using \(\scriptsize{\color{black}{D_1}}\) Twice or Once (KS measure)

Summary of variations on discount prior method

We have suggested a variation for the discount prior method so that there is less “double use” of the full set of current data in the final posterior.
- We partition D into two sets, using an initial subset, \(\scriptsize{D_1}\) to estimate \(\scriptsize{\alpha_0(D_0,D_1)}\).
- If \(\scriptsize{D_1}\) is considered a second prior dataset, then inference on \(\scriptsize{\theta}\) (after observing \(\scriptsize{D_2}\)) is legitimately Bayesian.
- We have not demonstrated how to decide what percentage of \(\scriptsize{D}\) should be used in \(\scriptsize{D_1}\).
We suggest alternative similarity measures to the “stochastic ordering” measure that may depend less on posterior variance, if needed.

Summary of results of the modified methods

Using the KS similarity measure (w/ Weibull disc function):
- Type I error rate was generally lower when an interim data percentage was used to estimate the borrowing fraction, as opposed to using all of the current data at the end of the study.
- Power was higher when using 100% of the current data (vs less than 100%) to estimate the amount of borrowing as well as to make an inference at the end of the study.
Using stochastic ordering measure (with different Weibull disc function) showed different patterns (lower type I error rates, but lower power).
Discarding \(\scriptsize{D_1}\) decreases efficiency (increases posterior variance), eliminates useful information, and can increase bias.
Simulations showed scant deflation of posterior variance after representing the information from \(\scriptsize{D_1}\) twice in the posterior distribution of \(\scriptsize{\theta}\).

Backup Slides

Limitations

The power prior method assumes patient-level exchangeability, which may be too strong. Covariates may be needed to calibrate the prior and current datasets even when the power parameter is less than 1.0.
One shouldn’t base the amount of borrowing solely on observed outcome responses because outcome responses may be influenced too much by sampling variability.
The KS statistic would not differentiate between similarity where the current data are slightly better than the prior and similarity where the current data are slightly worse.
- Stochastic ordering measure would differentiate these two situations
Ultimately, investigating design characteristics is necessary to ensure a reasonable trial design.

Type I Error Rate by \(\scriptsize{{{\theta }^{*}}}\) (null value) when \(\scriptsize{\bf{n = n_0 = 50}}\)

Samples from the Posterior of \(\scriptsize{\bf{\theta}}\) when \(\scriptsize{\bf{D_1} = 75\%}\) of D is Kept (top) and Discarded (bottom)

This appears to be due to increased posterior variance when you discard D1, especially when the sample size is lower. So, even if the posterior mean is closer to the null (top panel), because the variance is lower, more than 97.5% of the distribution falls into the H1 side, triggering a false rejection.

As an explanation, consider that when D1 is kept, post variance will be smaller than when D1 is discarded. So, even if the post mean is further away from alternative when D1 is kept versus when it is discarded, more of the posterior distribution could exceed the 97.5% posterior probability boundary due to lower posterior variance.

Figure shows an example of this when \(\scriptsize{{\theta^{*}}}\)= –0.25 (which corresponds to null value), and D1 represents 75% of the 50 current subjects. Top panel is a histogram of draws from the post distn of \(\scriptsize{{\theta}}\) when D1 is kept. The sample mean of the draws is –0.041. A vertical line at –0.25 separates draws consistent with null vs alternative. 98% of values are consistent with \(\scriptsize{H_1}\), triggering rejection of \(\scriptsize{H_0}\)

bottom panel shows posterior draws when D1 is discarded. Sample mean is 0.051 which is > –0.041. However, due to increased variance, only 89% of draws are consistent with \(\scriptsize{H_1}\).

\(\scriptsize{H_0}\)

\(\scriptsize{H_1}\)

\(\tiny{\bf{(\bar{\theta} = -0.04)}}\)

\(\tiny{\bf{(\bar{\theta} = +0.05)}}\)

Other Measures of Similarity in the Literature

1. Gravestock & Held (2017) maximize the marg. LH of \(\scriptsize{\alpha_0}\)

\[\small{\hat{\alpha}_0^{ML}= \underset{\alpha_0}{\mathrm{arg\,max}}=L(\alpha_0 | D_0, D) = \int L(\theta|D)\pi(\theta,\alpha_0|D_0) d\theta} \]

2. Nikolakopoulos et al. (2017) use the prior-predictive p-value:

\[\small{ppp(D_0, \hat{\alpha}_0) = 2 \min(P_{D|D_0, \hat{\alpha}_0}(T(D) \ge T(D^{obs})), \\ \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, P_{D|D_0, \hat{\alpha}_0}(T(D) \le T(D^{obs})))}\]

where \(\scriptsize{\hat{\alpha}_0 = \min[\max(\hat{\alpha}_0: ppp(D_0, \hat{\alpha}_0) \ge c), 1] }\).

Uses current data in the estimation of \(\scriptsize{\hat{\alpha}_0}\). Could be determined with interim current data (though not mentioned in paper).

Posterior SD of \(\scriptsize{\bf{{{\theta }}}}\) Keeping or Discarding \(\scriptsize{\color{black}{D_1}}\)

Bias of Posterior Mean of \(\scriptsize{\bf{{{\theta }}}}\) Keeping or Discarding \(\scriptsize{\bf{D_1}}\)

\(\scriptsize{\bf{\alpha _0(D_0,D_1)}}\) by \(\scriptsize{{{\theta }^{*}}}\) (prior mean = 0 and \(\scriptsize{\bf{n_0 = n}}\) = 50)

\(\scriptsize{\bf{\alpha _0(D_0,D_1)}}\) by \(\scriptsize{{{\theta }^{*}}}\) (prior mean = 0 and \(\scriptsize{\bf{n_0 = n}}\) = 25)

Type I Error Rate by \(\scriptsize{{{\theta }^{*}}}\) (null value) when \(\bf{\scriptsize{n = n_0 = 25}}\)

Power by \(\scriptsize{\bf{{{\theta }^{*}}}}\) when \(\scriptsize{\bf{n = n_0 = 50}}\)

Results from Nikolakopoulos et al.

\(\scriptsize{\bf{\alpha _0(D_0,D_1)}}\) by \(\scriptsize{{{\theta }_0}}\)
(\(\scriptsize{{{\theta }}}\) = 0 and \(\scriptsize{\bf{n_0 = n}}\) = 100)

Type I Error Rate by \(\scriptsize{{{\theta }_0}}\)
(\(\scriptsize{{{\theta }}}\) = 0 and \(\scriptsize{\bf{n_0 = n}}\) = 100)

Power by \(\scriptsize{{{\theta }^{*}}}\) for \(\scriptsize{{{H_0:\theta=0}}}\)
(\(\scriptsize{{{\theta_0}}}\) = 0.25 and \(\scriptsize{\bf{n_0 = n}}\) = 100)

Thanks to my co-authors:

Outline

Review of Conditional Power Prior

Prior Distribution given \(\scriptsize{D_0}\)

Posterior Distribution given \(\scriptsize{D}\)

Prior with two prior datasets \(\scriptsize{D_0}\) and \(\scriptsize{D_1}\)

Discount Prior Method (Haddad et al., 2017)

Prior given \(\scriptsize{D_0}\) (and \(\scriptsize{D}\))

Comments on Discount Prior Method

Determining the power parameter from an external existing dataset \(\scriptsize{D_1}\)

Eliminates double use, with \(\scriptsize{\alpha_0}\) determined a priori

Conditional Power Prior becomes:

\(\scriptsize{{{\alpha }_{0}}({{D}_{0}},{{D}_{1}})}\) is the proportion of down-weighting of the LH of \(\scriptsize{D_0}\) using information from both \(\scriptsize{D_0}\) and \(\scriptsize{D_1}\).

Determining the power parameter from interim current data (\(\scriptsize{D_1}\))

\(\scriptsize{{{\alpha }_{0}}({{D}_{0}},{{D}_{1}})}\) may not reflect a comparison between prior data (\(\scriptsize{D_0}\)) and current data, and \(\scriptsize{D_1}\) may not exist.

An alternative is to use interim information from the current data.

Suppose \(\scriptsize{D_1}\) is an initial subset of \(\scriptsize{D}\) of size \(\scriptsize{n_1}\) = %\(\scriptsize{n}\). Let \(\scriptsize{D=\left[{{D}_{1}},{{D}_{2}}\right]}\), then the likelihood of \(\scriptsize{D}\) is

Mathematically, we can consider \(\scriptsize{D_1}\) as a "second" prior dataset because it comes before \(\scriptsize{D_2}\) and after \(\scriptsize{D_0}\).

Determining the power parameter from interim current data (\(\scriptsize{D_1}\))

The resulting prior conditions on \(\scriptsize{D_0}\) and \(\scriptsize{D_1}\), but only down-weights the LH for \(\scriptsize{D_0}\).

Posterior given current data \(\scriptsize{D_2}\):

Yesterday's posterior is today's prior

Determining the power parameter from interim current data (\(\scriptsize{D_1}\))

Things to note:

Measuring Similarity of Prior and Current Data (Stochastic Ordering measure)

Hypothetical Posterior Distribution of \(\scriptsize{\theta}\) and \(\scriptsize{{{\theta }_{0}}}\) for Success Rate

Measuring Similarity of Prior and Current Data (Stochastic Ordering measure)

Hypothetical Posterior Distribution of \(\scriptsize{\theta}\) and \(\scriptsize{{{\theta }_{0}}}\) for Success Rate

Hypothetical Posterior Distribution of \(\scriptsize{\theta}\) and \(\scriptsize{{{\theta }_{0}}}\) for Success Rate

Hypothetical Posterior Distribution of \(\scriptsize{\theta}\) and \(\scriptsize{{{\theta }_{0}}}\) for Success Rate

Alternative Measure of (Dis)similarity: Kolmogorov-Smirnoff Statistic (KS)

Simulation Study

Prior data: \(\scriptsize{D_0 \text{~} N(0,1)}\) with sample size \(\scriptsize{n_0}\) = 100 (or 50).

Current data: Normal mean values, \(\scriptsize{{{\theta }^{*}}}\): \(\scriptsize{(– 1, – 0.75, – 0.5, }\) \(\scriptsize{– 0.25, – 0.1, 0)}\).

For each \(\scriptsize{{{\theta }^{*}}},\) we simulated 15,000 current data sets D from a \(\scriptsize{N({{\theta }^{*}}, 1)}\) distribution with size \(\scriptsize{n}\) = 100 (or 50).

The percent of \(\scriptsize{D}\) used to form \(\scriptsize{D_1}\): \(\scriptsize{100\%, 75\%, 50\%, 25\%}\).

Similiarity measure: KS statistic or stochastic ordering measure.

Simulation Study

Operating Characteristics: Type I error rate is computed under \(\scriptsize{H_0: \theta \le l}\), where \(\scriptsize{l \in (-1,..., -0.25, -0.10)}\).

\(\scriptsize{H_0}\) is rejected if posterior probability of \(\scriptsize{H_0}\) is less than 0.025.

Power is computed under \(\scriptsize{H_0:\theta \le -0.5}\) with values of \(\scriptsize{{{\theta }^{*}} \in (-0.35, -0.25, -0.10, 0)}\).

The maximum value of \(\scriptsize{\alpha_0}\) allowed is 1.0.

Form of \(\scriptsize{{{\alpha }_{0}}({{D}_{0}},{{D}_{1}})}\) for Simulations

Summary of variations on discount prior method

Summary of results of the modified methods

Limitations

Other Measures of Similarity in the Literature

1. Gravestock & Held (2017) maximize the marg. LH of \(\scriptsize{\alpha_0}\)

2. Nikolakopoulos et al. (2017) use the prior-predictive p-value:

where \(\scriptsize{\hat{\alpha}_0 = \min[\max(\hat{\alpha}_0: ppp(D_0, \hat{\alpha}_0) \ge c), 1] }\).

Suppose \(\scriptsize{D_1}\) is an initial subset of \(\scriptsize{D}\) of size \(\scriptsize{n_1}\) = %\(\scriptsize{n}\).
Let \(\scriptsize{D=\left[{{D}_{1}},{{D}_{2}}\right]}\), then the likelihood of \(\scriptsize{D}\) is

Alternative Measure of (Dis)similarity:
Kolmogorov-Smirnoff Statistic (KS)

For each \(\scriptsize{{{\theta }^{}}},\) we simulated 15,000 current data sets D from a \(\scriptsize{N({{\theta }^{}}, 1)}\) distribution with size \(\scriptsize{n}\) = 100 (or 50).